-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Object Detection & Semantic Segmentation #136
Object Detection & Semantic Segmentation #136
Conversation
e97b1ae
to
b6c66cf
Compare
d81faca
to
e165da6
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome work! 🔥 Documentation is quite thorough and I like the way the operators are extended to object detection and segmentation. Comments are mostly typos and ways to improve the clarity of the explanations, but it really shoudn't take long to resolve them ;)
d0d08dd
to
ea1284d
Compare
ea1284d
to
b699ad3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Again, awesome work!
Features
Adapt code to adopt new features
New operators will be added, hence, to clarify the code, operators and the relative operations were divided between
operators.py
andoperators_operations
in 7cc91cc. Naturally, the corresponding tests were adapted in 285f7dc.Semantic Segmentation
After long debates, with several persons on the team, we agreed on what we may want to explain in semantic segmentation and the relative functions to explain. All those functions were merged into the new
semantic_segmentation_operator
proposed in 42dbe70 and tested in fd15ed3. It supports perturbation-based and gradient-based methods. Furthermore, the semantic segmentation tutorial is on a pytorch model, thus, it is compatible with theTorchWrapper
.Furthermore, for this operator, the
targets
parameter is complicated to design, therefore, a set of functions was necessary. They were placed insegmentation.py
in a newutils_functions/
directory in 90f7f9b, with the corresponding tests in e0ba290, and the documentation integrated to semantic segmentation documentation page.Note, we detected that it is possible to simplify this operator, it will be done when taking into account the remarks on this PR.
Object Detection
Following Semantic Segmentation, a new operator:
object_detection_operator
was introduced in 42dbe70 and tested in fd15ed3. It also supports perturbation-based, gradient-based methods, and theTorchWrapper
as shown in the object detection tutorial.Some functions used by the
object_detection_operator
were initially in theattributions/object_detector.py
file and moved to theutils_functions/object_detection.py
file in 90f7f9b and tested in e0ba290. It was moved because theattributions/object_detector.py
file should depend onoperators.py
and not the opposite. Furthermore,attributions/object_detector.py
will be removed in future versions, see next section.Note that a feature was requested: "We should be able to explain the different parts of an object detection prediction (i.e. box position, proba, ad class". This feature was added and can be specified by adding either
" box position"
," box proba"
, or" box class"
to the initial"object detection"
value of theoperator
parameter.BoundingBoxExplainer
is now deprecatedThe initial way to apply Xplique to object detection was to use the
BoundingBoxExplainer
wrapper. However, with the API update, it was natural for the object detection API to match. Furthermore, no documentation was made previously, thereforeBoundingBoxExplainer
is now deprecated through 37d9214. Nonetheless, the new implementation of this class is just a call to theobject_detection_operator
which was tested in 26ee07a.Finally, this is the first case of deprecation in Xplique, the choice was made to use the
deprecated
library which adds a dependency. This choice can be discussed, but this was the only one tested that showed the indicated deprecation warning message. This was added in d39ed81.Documentation
Merge model, operator, and API description page into one
First, the
docs/
directory architecture did not match the documentation architecture, it was confusing for me, hence I modifieddocs/
to match the documentation architecture in f5ac35e.As @fel-thomas highlighted in #132 remarks, the documentation was too divided, furthermore, a lot of information was redundant between those pages and they were interdependent. Hence the choice was made to merge the model, operator, and API description page into one in c7483d8.
Create task related pages
Two tasks (Object Detection and Semantic Segmentation) were introduced in the documentation, their complexity induced a specific documentation page. However, it was not consistent to have documentation pages only for those two tasks. Therefore information about Classification and regression was extracted from the common API page to create two other new task-specific pages. Finally, four task-specific were introduced to the documentation in d6a72d6.
Adapt other documentation files to changes
Finally, other documentation files were adapted to the new documentation links in 731d7ac. On those files, the choice was made to add the operator specification in all snippets of code related to attribution methods. It may seem heavy but I think it improves clarity.
Others
There was a pylint error in maco file, pylint does not like
isinstance(variable, Callable)
hence it was changed tohastattr(variable, "__call__")
in e817170.Then, some typo were corrected in b699ad3.